Skip to content

Reference Models Centrally, set OpenAI Model to gpt-5-nano and Updated Spec Defaults #4608

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 11 commits into
base: v1.15
Choose a base branch
from
Open

Reference Models Centrally, set OpenAI Model to gpt-5-nano and Updated Spec Defaults #4608

wants to merge 11 commits into from
+126 −14

Conversation

giterinhub
Copy link

@giterinhub giterinhub commented Apr 5, 2025 •
edited
Loading

Feat: Add new conversation components and update model defaults

This pull request introduces documentation for two new conversation components (GoogleAI and Ollama) and updates the default LLM models for several existing components to newer versions.

A key change is that the default model for each conversation component can now be configured at runtime via a dedicated environment variable.

Summary of Changes:

  • New Conversation Components:

    • Added documentation for the conversation.googleai component, which defaults to the gemini-2.5-flash-lite model.
    • Added documentation for the conversation.ollama component, which defaults to the llama3.2:latest model.

  • Default Model Updates:

    • Anthropic: The default model has been updated from claude-3-5-sonnet-20240620 to claude-sonnet-4-20250514.
    • Hugging Face: The default model has been updated from meta-llama/Meta-Llama-3-8B to deepseek-ai/DeepSeek-R1-Distill-Qwen-32B.
    • OpenAI: The default model has been updated from gpt-4-turbo to gpt-5-nano.
    • Mistral: The default model remains open-mistral-7b.

  • Configuration via Environment Variables:

    • Each conversation component's default model can now be overridden using a specific environment variable (e.g., OPENAI_MODEL, GOOGLEAI_MODEL, etc.).
    • The central environment variable reference page has been updated to include documentation for all new model environment variables:
      • ANTHROPIC_MODEL
      • GOOGLEAI_MODEL
      • HUGGINGFACE_MODEL
      • MISTRAL_MODEL
      • OLLAMA_MODEL
      • OPENAI_MODEL

@giterinhub giterinhub requested review from a team as code owners April 5, 2025 21:53
@msfussell
Copy link
Member

@giterinhub - Is there a corresponding code issue tracking this that you are aware of?

@msfussell msfussell added the waiting-on-code-pr The code PR needs to be merged before the docs are updated label Apr 13, 2025
@msfussell msfussell added this to the 1.16 milestone Apr 13, 2025
@giterinhub giterinhub mentioned this pull request Apr 14, 2025
Merged
8 tasks
@giterinhub
Copy link
Author

Created dapr/components-contrib#3792

@giterinhub
Copy link
Author

@giterinhub - Is there a corresponding code issue tracking this that you are aware of?

No issue I'm aware of, just created the PR. Thanks for the good time at the Boom Battle Mark!

@giterinhub giterinhub closed this Aug 29, 2025
@giterinhub giterinhub changed the title Set OpenAI Model to gpt-4o-mini and Updated Spec Defaults Reference Models Centrally, set OpenAI Model to gpt-5-nano and Updated Spec Defaults Aug 29, 2025
@giterinhub giterinhub reopened this Aug 29, 2025
@msfussell msfussell removed the waiting-on-code-pr The code PR needs to be merged before the docs are updated label Sep 13, 2025
@msfussell
Copy link
Member

@giterinhub - Can you review this PR now the code is in. In particular can you address
@sicoyle @bibryam - Please review

  1. How is the AZURE_OPENAI_MODEL env used and by which component? How does this relate to the apiType field here https://v1-16.docs.dapr.io/reference/components-reference/supported-conversation/openai/ used to set Azure usage?
  2. Why have you included the OLLAMA model, there is an existing MD file, I presume this is not needed or update the existing one?
  3. Can you update the https://v1-16.docs.dapr.io/reference/environment/ list with the list of these 7 env variables with a description of each one.
  4. Should there be an env var for AWS Bedrock and if not why?

@msfussell msfussell mentioned this pull request Sep 13, 2025
Open
sicoyle
sicoyle reviewed Sep 15, 2025
View reviewed changes
Copy link
Contributor

@sicoyle sicoyle left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

few comments for this. Thank you for following up with the corresponding docs PR for your changes 🙌 🚀

@giterinhub
Copy link
Author

Thanks @sicoyle !

sicoyle
sicoyle approved these changes Sep 16, 2025
View reviewed changes
Copy link
Contributor

@sicoyle sicoyle left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM - thank you!

@sicoyle
Copy link
Contributor

sicoyle commented Sep 16, 2025

@giterinhub any chance you can correct the DCO step on the PR? If you click in on the build step it has the cmds you can run to make it clean and green :)

marcduiker
marcduiker approved these changes Sep 16, 2025
View reviewed changes
Copy link
Contributor

@marcduiker marcduiker left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

marcduiker and others added 3 commits September 16, 2025 14:20
@marcduiker
Add kapa.ai script (dapr#4853)
a0d6cb2 
Signed-off-by: Marc Duiker <[email protected]>
@TechyMT
fix: update Redis documentation to reference latest Sentinel document…
de42393 
…ation links (dapr#4860)

Modified multiple Redis-related markdown files to update the links for the Sentinel master name documentation from the old URL to the latest version. This ensures users have access to the most current information regarding Redis Sentinel configuration.

Signed-off-by: Mustafa <[email protected]>
@giterinhub
docs(conversation): align runtime model env vars & update component e…
96eb17c 
…xamples

- Add and document runtime env vars: OPENAI_MODEL, AZURE_OPENAI_MODEL, ANTHROPIC_MODEL, GOOGLEAI_MODEL, MISTRAL_MODEL, HUGGINGFACE_MODEL, OLLAMA_MODEL (with defaults)
- Update environment reference page with the new env var names and defaults
- Replace env-var placeholders in component YAML examples with literal default model names for OpenAI, GoogleAI, Anthropic, Mistral, Hugging Face and Ollama
- Add "(configurable via '<ENV>' environment variable)" to each provider's Spec metadata table to show override option
- Add Azure OpenAI usage section to `conversation.openai` doc (how to use `apiType: azure` and `AZURE_OPENAI_MODEL`)
- Clarify AWS Bedrock uses standard AWS auth (no Bedrock-specific env var added)
- Fix small markdown lint issues (trailing newlines/whitespace)

Signed-off-by: Erin La <[email protected]>
giterinhub and others added 5 commits September 16, 2025 14:20
@msfussell msfussell modified the milestones: 1.16, 1.17 Sep 16, 2025
@msfussell msfussell added the waiting-on-code-pr The code PR needs to be merged before the docs are updated label Sep 16, 2025
@msfussell
Copy link
Member

Need to wait on this PR from main to branch dapr/components-contrib#4029

@bibryam
Copy link
Contributor

bibryam commented Sep 16, 2025

@msfussell

  1. How is the AZURE_OPENAI_MODEL env used and by which component? How does this relate to the apiType field here https://v1-16.docs.dapr.io/reference/components-reference/supported-conversation/openai/ used to set Azure usage?

When apiType = "azure": Uses AZURE_OPENAI_MODEL environment variable (defaults to "gpt-4.1-nano")
When apiType is not "azure": Uses OPENAI_MODEL environment variable (defaults to "gpt-5-nano")

  1. Why have you included the OLLAMA model, there is an existing MD file, I presume this is not needed or update the existing one?

I don't see additional MD file, maybe it was removed.

  1. Can you update the https://v1-16.docs.dapr.io/reference/environment/ list with the list of these 7 env variables with a description of each one.
    Seems that is done
  1. Should there be an env var for AWS Bedrock and if not why?

IMO Bedrock should follow the same pattern and support env variable for model override.

Apart from these, this is a great PR (we need a similar PR for API KEY separately.

@giterinhub
Copy link
Author

Mentioned already that I like to have AZURE_OPENAI_MODEL and OPENAI_MODEL separately @bibryam I had to remove the entry for AZURE_OPENAI_MODEL as Sam wanted it gone ;)

@sicoyle
Copy link
Contributor

sicoyle commented Sep 16, 2025 •
edited
Loading

Mentioned already that I like to have AZURE_OPENAI_MODEL and OPENAI_MODEL separately @bibryam I had to remove the entry for AZURE_OPENAI_MODEL as Sam wanted it gone ;)

AZURE_OPENAI_MODEL is not a thing... all components that use the conversation.openai type will just use the env var of OPENAI_MODEL if they choose to use the env var to set their model. There are several components, not just azure that are openai compatible, so we don't want a ton of env vars for each of them. Since they all can just use the openai component, then they can all use the openai env var. Examples include: Minimax, Qwen, Ernie, Azure, etc.

Editting to add a few clarifications:

There is logic under the hood that checks if apiType is set to azure. In that case, the appropriate Azure model is chosen instead of defaulting to the generic OpenAI model.

@bibryam Please create a separate issue if you’d like the API_KEY to also be set via an env var. That said, should all metadata fields be configurable this way? It feels like a lot—so where do we draw the line?

Looking back at the contrib PR, it appears we missed Bedrock. This shouldn’t block the current PR and can be added later, but for now, it will be missing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@marcduiker marcduiker marcduiker approved these changes

+1 more reviewer

@sicoyle sicoyle sicoyle approved these changes

Reviewers whose approvals may not affect merge requirements
Assignees

@giterinhub giterinhub

Labels
waiting-on-code-pr The code PR needs to be merged before the docs are updated
Projects
None yet
Milestone
1.17
Development

Successfully merging this pull request may close these issues.
6 participants
@giterinhub @msfussell @sicoyle @bibryam @marcduiker @TechyMT